Skip to content

Add all-users validation backfill - #521

Merged
msaroufim merged 1 commit into
mainfrom
agent/cholesky-validate-all-users
Jul 26, 2026
Merged

Add all-users validation backfill#521
msaroufim merged 1 commit into
mainfrom
agent/cholesky-validate-all-users

Conversation

@msaroufim

@msaroufim msaroufim commented Jul 26, 2026

Copy link
Copy Markdown
Member

Summary

  • add --all-users to the existing kernelbot-admin validate-top10 command
  • pass the manual scope through the admin API and validation service
  • select every ranked user's current best submission for an all-users sweep
  • keep scheduled sweeps capped at the top 10 users and concurrency capped at two
  • require no Kernelboard change; existing validation joins already work for every rank

Why

Cholesky currently has 93 ranked users. This provides a controlled one-time backfill so every current best submission can receive a validation annotation without expanding the nightly workload.

Validation

  • PYTEST_MODAL_ENV=pytest uv run pytest tests/ -v — 260 passed, 7 skipped
  • focused all-users/API/CLI coverage — 11 passed
  • all hosted lint, unit, GitHub integration, and Modal integration checks passed
  • Ruff on all changed Python files
  • git diff --check
  • CLI help confirms --all-users

@github-actions

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  src/libkernelbot
  application_validation.py 107-116, 302
  utils.py
Project Total  

This report was generated by python-coverage-comment-action

@msaroufim
msaroufim merged commit 61cb1af into main Jul 26, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant